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We present a search for the production of a heavy gauge boson, W' , that decays to third-generation 
quarks, by the DO Collaboration in pp collisions at y/s = 1.96 TeV. We set 95% confidence level 
upper limits on the production cross section times branching fraction. For the first time, we set 
limits for arbitrary combinations of left- and right-handed couplings of the W' boson to fermions. 
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present. The limit for right-handed couplings improves for M(W') < miypi) to M(W') > 890 GeV. 
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In the standard model (SM), the charged weak current 
is mediated by W bosons. Many models of physics 
beyond the standard model predict the existence of 
additional charged bosons, generally called W, that are 
more massive than the W boson of the standard model. 
The chiral structure of the couplings of the W boson to 
fermions depends on the details of the model. 

In models of universal extra dimensions the W is 
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the lowest Kaluza-Klein excitation of the W boson. It 
therefore has the same couplings to fermions as the W 
boson and couples exclusively to left-handed fermions. 
Left-right symmetric models [2J add an additional group 
SU(2) R to the SM. Thus there are two bosons, W L 
and Wji, associated with the SM SU(2)l group and the 
additional SU{2)r group that would mix to form the 
physical states. One of these physical states is identified 
with the W boson. Since the W boson is known experi- 
mentally to couple only to left-handed fermions, the 
other state, W, would have to almost exclusively couple 
to right-handed fermions. If there exists a light right- 
handed neutrino i/r, the decay W — > (.vr (where I is a 
charged lepton) is allowed. Otherwise, the W' can only 
decay to quarks, making it harder to observe. 

Independent of specific models, the most general, 
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lowest-dimension Lagrangian for the interaction of a W 
boson with fermion fields / is given by Q 

C = ^/ i7 „ (4 (1 + 7 5 ) + 4 (1 " 7 5 )) W'»f j+ h.c, 



(1) 

where a^ fl are the left /right-handed couplings of the W 
boson to the fermion doublet f t and fj and g w is the weak 
coupling constant of the SM. If the fermions are quarks, 
the Vij are the elements of the CKM matrix for quark 
flavors i, j. If the fermions are leptons, the Vij are the 
elements of a 3 x 3 identity matrix. 

In this Letter, we report on a search for a W boson 
that decays to a top quark and an anti-bottom quark 
(or charge conjugates) [iMf using 2.3 fb _1 of integrated 
luminosity from proton- antiproton collisions at \fs — 
1.96 TeV, accumulated by the DO experiment |6j at the 
Fermilab Tevatron Collider between 2002 and 2007. This 
is the same dataset used in the observation of single top 
quark production by the DO Collaboration [7[. We use 
the notation tb to represent the sum of the tb and the 
tb final states. The tb decay channel is sensitive to W 
bosons with left- and right-handed couplings regardless 
of the existence of a light right-handed neutrino. For left- 
handed couplings, the process pp — > W + X — >• tb + X 
interferes with SM single top quark production pp — ► 
W* +X -> tb + X. 

We set limits on the production cross section 
for such W' bosons and translate them into limits 
on left- and right-handed gauge couplings. We 
have previously excluded W bosons with masses of 
M(W') < 731 GeV for purely left-handed couplings and 
M{W) < 739 (768) GeV for the case of purely right- 
handed couplings with (without) a light right-handed 
neutrino [8j. The CDF Collaboration has published 
similar limits [qJ, neglecting the interference with SM 
single top quark production. Relative to our earlier 
publication, we use more than twice the integrated 
luminosity and improve the overall sensitivity of the 
analysis to a W boson signal by using boosted decision 
trees (BDT). In Ref. j7j it was demonstrated that a single 
top quark signal at the level of a few percent was clearly 
identifiable by the BDT technique. For the first time, we 
perform a general analysis of the left- and right-handed 
gauge couplings a^,afj of the W boson, < a^- R < 1. 
For comparison with the earlier work, we also quote 
results for three special cases: (1) purely left-handed 



couplings, de- 
couplings, afj 

of left- and right-handed couplings, a. 



- a,afj = Vi,j; (2) purely right-handed 
0, afj = a Vi, j; and (3) an equal mixture 



afj = a Vi,j. 

Finally, we set limits on the mass of such W bosons 
under the assumption of a "SM-like" coupling strength, 
i.e., a = l. 

We search for events in which a W boson is produced 
and subsequently decays into tb. The top quark decays 
to Wb and the W boson is required to decay to ev or jxv. 



W boson decays into r-leptons or quarks suffer from an 
overwhelming background. Thus our final state contains 
an electron or muon, missing transverse momentum from 
the undetected neutrino, and two jets from the fragmen- 
tation of the two 6-quarks. Additional jets may arise from 
initial- or final-state radiation. We acquire these events 
using a logical OR of many trigger conditions that require 
a combination of jets, electrons and muons. 

Object selections and definitions are identical to 
Ref. Ml- Events are selected with exactly one isolated 
electron with pr > 15 GeV for two-jet events or px > 
20 GeV for three- and four-jet events, and \rj\ < 1.1 or 
one isolated muon with pt > 15 GeV and \rj\ < 2.0, where 
where t] is the pseudorapidity. The leading jet must have 
Pt > 25 GeV and there must be a total of two, three, 
or four jets with pt > 15 GeV and \rj\ < 3.4. Jets are 
defined using a cone algorithm 10] with a cone radius 
A1Z = \J (A</>) 2 + (Ay) 2 =0.5, where y is the rapidity. 
Missing pt is required to be greater than 20 (25) GeV 
for two-jet (three- or four-jet) events. To enhance the 
signal content of the selected data sample, one or two of 
the jets are required to be identified as originating from 
the fragmentation of a & quark llj . We call these jets b- 
tagged. In order to minimize the effect of interference 
with SM single top quark production, we require the 
invariant mass, y§, of the reconstructed charged lepton, 
the jets, and missing pr to be greater than 400 GeV. 

We simulate the complete chain of W decays, taking 
into account finite widths and spin correlations between 
the production of resonance states and their decay using 
the SINGLETOP Monte Carlo package (l2j based on the 
SINGLETOP event generator 13J. We set the top quark 
mass to 172.5 GeV; the factorization scale is set to 
M(W) and we use the CTEQ6M parton distribution 
functions 14]. The SINGLETOP generator reproduces 
the next-to-leading-order (NLO) kinematic distributions, 
and the interference term between the W boson signal 
and the SM W boson. The width of the W boson varies 
between 20 and 30 GeV for W masses between 600 and 
900 GeV It is 25% smaller without a light right- 

handed neutrino. The width does not have a significant 
effect on our search because it is smaller than the 
detector resolution. Therefore, the only relevant effect 
of a massive neutrino is the larger branching fraction for 
W 1 — > tb. Unless mentioned otherwise we will assume in 
the following that there is a right-handed neutrino with 
M(W) > mivfs). In this case the limits for right-handed 
couplings will be weaker than for M(W') < m(i/fj) 
because of the smaller branching fraction for W' — > tb. 
In the absence of interference between W and W bosons 
and in the presence of a light right-handed neutrino, 
there is no difference between W' bosons with left- and 
right-handed couplings for our search. To probe for W' 
bosons with masses above the currently published limits, 
we simulate W bosons at nine mass values from 600 to 
1000 GeV. 
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Background yields are estimated using both Monte 
Carlo samples and data. The procedure is identical to 
that used in Ref. [7] . The multijet background arises from 
events in which a jet mimics the signature of an isolated 
charged lepton. It is modeled using control data samples 
with non isolated leptons. The dominant background for 
our search originates from W+jets production. Smaller 
sources of backgrounds are from ti pair, i-channel single 
top quark (tqb), diboson, and Z+jets events. The 
diboson (WW, WZ, ZZ) backgrounds are modeled 
using pythia 15]. The other background processes 
are modeled using the ALPGEN 16| event generator and 
subsequently hadronized using pythia. The fraction 
of VK+heavy flavor events (Wbb, and Wee) is obtained 
by scaling the cross sections calculated by ALPGEN 
to the NLO cross section by a factor of 1.47. An 
additional correction factor of 0.95 ± 0.13, derived from 
data samples with different number of 6-tagged jets, 
is applied. The VKc+light parton cross sections are 
scaled by a factor of 1.38. All processes except the 
W^+jets and multijets background are normalized to 
their expected cross sections [l7|, LH|. The W+jets 
and multijets yields are obtained by normalizing to the 
data sample after subtracting all other backgrounds and 
before selecting events based on 6-tagged jets. At this 
point in the selection, the data are completely dominated 
by background and any contamination from a W' boson 
signal is negligible. 

The sensitivity of the search is maximized by dividing 
the data into 24 independent channels based on lepton 
flavor (e, fi), jet multiplicity (2, 3, 4), number of b- 
tagged jets (1, 2), and two data collection periods, to 
take into account different signal acceptances and signal- 
to-background ratios [7(. After applying all selections, we 
find the event yields for data and backgrounds as shown 
in Table QJ The requirement \fi > 400 GeV accepts 
most of the W' contribution but eliminates most of the 
W boson contribution. As the mass of the W' boson 
increases, its contribution to the tb final state decreases 
relative to that of the W boson, and therefore, the 
efficiency of this requirement decreases between 3.5% and 
0.5% for W' boson masses between 600 and 1000 GeV. 

TABLE I: Data and SM background event yields with 
systematic uncertainties. 



Process 


Events 


tqb 


26.4 ± 2.5 


ti 


424.7 ± 58.4 


W+jets 


279.5 ± 18.3 


Z+jets 


26.0 ± 3.2 


Dibosons 


13.0 ± 1.6 


Multijets 


60.5 ± 10.8 


Total backf 


;round 830 ± 62 


Data 


831 



At this stage, the expected W' boson signal would 



constitute only a small fraction of the selected data 
sample. To improve discrimination, we calculate a 
multivariate discriminant based on BDTs that separates 
the W' boson signal from background and thus enhances 
the probability to observe W' boson production. We 
compute the discriminant independently in each of the 
24 channels. The input variables used to train the BDT 
discriminants take into account the kinematic properties 
and angular correlations of the reconstructed objects 
and the topology of the event. A BDT is trained 
for each W' boson mass value using the Monte Carlo 
sample generated with a L — a R = 1. The final 
discriminant for all 24 channels combined is shown in 
Fig. [TJa). To represent the discriminant distribution 
expected from an arbitrary combination of couplings, 
we combine samples of W' decays generated with left- 
handed, right-handed and mixed couplings and SM s- 
channel tb production based on theoretical expectations 
from Ref [3j. Background events preferentially populate 
the low discriminant region whereas signal events are 
clustered towards high discriminant values. We observe 
good agreement between background prediction and data 
in all channels. The data show no deviation from 
background only expectations and we set 95% C.L. upper 
limits on the cross section for the process W' — > tb — > 
tvbb using Bayesian statistics (l9j . A Poisson distri- 
bution for the observed counts and a flat non-negative 
prior probability for the signal cross section are assumed. 
Systematic uncertainties are taken into account with 
Gaussian priors. 

The systematic uncertainties that affect the signal 
and background models are described in Ref. [20| . The 
largest sources of systematic uncertainties are the jet 
energy scale calibration and the modeling of 6-tagging 
performance. Smaller uncertainties arise from the finite 
size of the MC samples, the corrections of the flavor 
composition of W+jets events, and from the normal- 
ization of the background. The total uncertainty in 
the background yield varies between 8% and 10% for 
the different channels. In determining the effect of 
the uncertainties from jet energy scale calibration, 6-tag 
modeling, and W+jets modeling, we take into account 
changes in the shape of the discriminant distributions in 
addition to normalization effects. 

The cross section for single top quark production in the 
presence of a W' boson for any set of coupling values can 
be written in terms of the cross sections for purely 
left-handed couplings (a L ,a R ) — (1,0), er^ for purely 
right-handed couplings (a L ,a R ) = (0,1), glr. for mixed 
couplings (a L ,a R ) = (1,1), and asM for SM couplings 
(a L ,a R ) = (0, 0). It is given by: 

O = (TSM + &ud a \b {vl - (Jr) (2) 
+ ((a^affc) 2 + (a^afb) 2 ) {?r ~ <?sm) 

1 / 2 2 \ 

+ 2 {( a nd a tb) + (aud a tb) ) (VLR -VL-VR + S m) ■ 
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The predicted cross section for SM single top quark 
production through s-channel W boson exchange is 
&sai = 1.12 ± 0.05 pb [Hj]. The predicted cross sections 
ctr, and a L ji as a function of W boson mass are 
listed in Table [TTJ We assume that the couplings to 
first generation quarks, a u d, which are important for the 
production of the W boson, and the couplings to third 
generation quarks, a t b, which are important for the decay 
of the W boson, are equal. For given values of a L and 
a R , the distributions are obtained by combining the four 
samples according to Eq. @. 

We vary both a L and a R between and 1 in steps of 
0.1, for each W boson mass value. We consider each of 
these 121 combinations of a L , a , and M(W) as a model. 
For each of these models, the ratio of the s-channel and 
the W cross sections is fixed. We then vary the total 
cross section to determine the expected and observed 95% 
C.L. upper limits. For the three special cases (a L ,a R ) 
: 1 . 0), (0, 1), and (1, 1), the measurements are listed along 
with the theoretical cross sections in Table [Til For small 
W masses, the cross section limits drop below the cross 
section for SM single top quark production in the s- 
channel, because the data do not accommodate the large 
W component that would have to be present for these 
models. The cross section limit reflects the maximum 
total cross section that the data can accommodate for the 
ratio of s-channel and W production in the particular 
model. We can now assume values for any two of the 
three parameters, a L , a R , and M(W), and interpolate 
the cross section limit in the third parameter value. 
The value of the third parameter at which the cross 
section limit equals the theory cross section Q represents 
the limit on the third parameter. Figure HJb) shows 
contours of lower limits for the W boson mass in the 
(a L ,a R ) plane, and Fig. [ljc) shows contours of upper 
limits for the coupling a L in the (a R , M(W')) plane for 
M(W') > m(vn). Fig. [2]shows the cross section limits for 
the three special cases. As the W boson mass increases, 
the selection efficiency decreases and the upper limits on 
the cross section increase. 

In conclusion, we have carried out a search for a 
massive charged gauge boson, W , that decays to tb. We 
considered a model-independent approach in which the 
W' boson may couple to fermions with any combination 
of left- and right-handed couplings. We do not observe 
any deviations from SM expectations and set upper 
limits on the cross section for the production of W 
bosons. We compare upper limits to theory cross 
sections to extract the following limits for M(W') > 
m(v R ): M{W) > 863 GeV for purely left-handed 
couplings, M(W) > 885 GeV for purely right-handed 
couplings, and M(W) > 916 GeV if both left- and right- 
handed couplings are present. The limit for right-handed 
couplings improves for M(W) < m{un) to M(W) > 
890 GeV. These mass limits improve previously published 
results by more than 100 GeV. 



TABLE II: NLO production cross section times branching 
fraction, o(pp — > W/W' — > tb), and expected and observed 
95% C.L. upper limits for different W' boson masses, 
assuming M(W') > m(i/fj), in pb. 



M(W) 


(a L , 


a H ) = 


(0,1) 


(a L , 


a H ) = 


(1,0) 


a") = 


(1,1) 


(GeV) 


OR 


obs 


exp 


Oh 


obs 


exp 


OLR 


obs 


exp 


600 


3.22 


0.12 


0.31 


2.16 


0.09 


0.22 


4.13 


0.09 


0.21 


650 


2.37 0.16 


0.37 


1.43 


0.10 


0.23 


2.62 


0.10 


0.26 


700 


1.86 


0.32 


0.46 


1.03 


0.18 


0.26 


1.74 


0.15 


0.23 


750 


1.56 


0.60 


0.64 


0.80 


0.37 


0.36 


1.24 


0.27 


0.28 


800 


1.38 


0.64 


0.92 


0.68 


0.39 


0.51 


0.95 


0.24 


0.34 


850 


1.28 


0.85 


1.44 


0.61 


0.48 


0.78 


0.78 


0.28 


0.46 


900 


1.21 


1.39 


2.06 


0.58 


0.93 


1.23 


0.69 


0.56 


0.57 


950 


1.18 


2.23 


2.81 


0.57 


1.50 


2.05 


0.64 


0.90 


1.17 


1000 


1.15 


2.39 


3.22 


0.57 


2.23 


3.06 


0.62 


1.24 


1.80 
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Boosted Decision Trees Output a L a R 



FIG. 1: (a) Distribution of the discriminant for data (points) compared to the background model summed over all analysis 
channels (filled histograms) and expected W' boson signal with mixed couplings (a L ,a R ) = (1,1) (open histogram), (b) 
Contour plots of 95% C.L. lower limits on M(W') in the (a L , a R ) plane, and (c) 95% C.L. upper limits on the coupling a L in 
the (a R ,M(W)) plane. Here M(W) > m{v R ). 




M(W') [GeV] M(W') [GeV] M(W') [GeV] 



FIG. 2: Expected and observed upper limits and theory prediction for the production cross section (the shaded band indicates 
the uncertainty ||) for W' bosons with (a) left-handed couplings (a L ,a R ) — (1,0), (b) right-handed couplings (a L ,a R ) — (0, 1) 
and, (c) mixed couplings (a L , a R ) = (1, 1) as a function of the W' boson mass. Here M(W') > m{yjt). 



[14] J. Pumplin et aL, J. High Energy Phys. 07, 012 (2002). 
[15] T. Sjostrand, S. Mrenna, and P. Skands, J. High Energy 

Phys. 05 (2006) 026. We used "pythia 6.409". 
[16] M. L. Mangano et ai, J. High Energy Phys. 07, 001 

(2003). 

[17] N. Kidonakis, and R. Vogt, Phys. Rev. D 68, 114014 
(2003); J.M. Campbell, and R.K. Ellis, Phys. Rev. D 65, 
113007 (2002); R. Hamberg, W.L. van Neerven, and 



W.B. Kilgore, Nucl. Phys. B359, 343 (1991) and erratum 

in B644, 403 (2002). 
[18] N. Kidonakis, Phys. Rev. D 74, 114012 (2006). 
[19] I. Bertram et ai, Fermilab-TM-2104, (2000), 

unpublished, and references therein. 
[20] DO Collaboration, V.M. Abazov et al, Phys. Rev. D 78, 

012005 (2008). 



